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FOREWORD 


This  project  was  conducted  for  the  NATO  SEASPARROW  Surface  Missile  System 
(NSSMS)  project  office  (SEA-06P)  in  response  to  a  request  by  Commander,  Naval  Sea 
Systems  Command  (NAVSEA  0141(3)).  The  project  resulted  from  initial  efforts  that  were 
conducted  under  the  advanced  development  task  area  Z0S28-PN  (Enlisted  Personnel 
Individualized  Career  System  (EPICS)). 

NSSMS  ordnance  publications  were  analyzed  according  to  human  factors  document 
design  criteria  and  fleet  input.  The  results  of  this  effort  are  intended  for  use  by  NSSMS 
project  office  to  reformat  the  NSSMS  ordnance  publications. 

J.  W.  RENARD  JAMES  W.  TWEEDDALE 

Captain,  U.S.  Navy  Technical  Director 

Commanding  Officer 
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INTRODUCTION 


Problem 

Concern  expressed  by  fleet  commanders  in  chief  regarding  inadequate  technical 
documentation  of  the  NATO  SEASPARROW  Surface  Missile  System  (NSSMS)  and  the 
extensive  cost  of  updating  the  technical  manuals  in  their  current  configurations  prompted 
the  NSSMS  project  office  to  consider  the  effectiveness  of  the  NSSMS  technical  publica¬ 
tions.  With  the  Naval  Ship  Weapon  Systems  Engineering  Station  (NSWSES)  assuring 
technical  accuracy  of  the  NSSMS  data,  a  human  factors  review  was  initiated  to  identify 
content  and  organizational  discrepancies  in  the  technical  information. 

Purpose 

The  purpose  of  this  effort  was  to  identify  specific  problem  areas  in  the  NSSMS 
ordnance  publications  (OPs)  and  make  recommendations  for  improving  the  usability  of  the 
NSSMS  technical  data. 


Background 


The  technical  data  for  the  NSSMS  comprise  48  OPs  in  the  following  seven  major 
divisions:  (l)  overall  system,  (2)  fire  control  subsystem,  (3)  launcher  subsystem,  (4) 
computer,  (5)  signal  data  converter,  (6)  low-level  light  television,  and  (7)  system 
evaluation  trainer. 


Booher  and  Mroz  (1981)  reviewed  the  NSSMS  documentation  and  found  that  most  of 
the  technical  data  in  the  OPs  were  oriented  toward  engineers  rather  them  technicians. 
They  concluded  that  the  large  NSSMS  technical  data  base  was  poorly  organized  and 
indexed  for  the  user.  It  was  not  readily  apparent  from  the  OPs  if  a  top-down  systems 
approach  had  been  used  to  organize  and  sort  the  technical  data. 


APPROACH 

The  NSSMS  OPs  were  reviewed  using  human  factors  criteria  to  identify  information 
presentation  discrepancies.  Each  category  of  information  (e.g.,  function  diagrams, 
schematics,  troubleshooting  tables,  etc.)  was  reviewed  and  examined  for  completeness  and 
overall  usability.  Samples  of  the  NSSMS  OPs  were  analyzed  from  the  user's  viewpoint  to 
identify  any  technical  content  areas  that  are  incompatible  with  user  needs.  To  determine 
usability,  the  technical  data  were  assessed  according  to  (1)  user  access  schemes  required 
to  locate  specific  sets  of  data,  (2)  completeness  of  the  information  for  fault  isolation,  (3) 
consistency  of  data  organization  across  and  within  subsystems,  and  (4)  comprehensibility 
requirements  of  the  user  population. 

The  organization  of  the  troubleshooting  data  was  examined  to  determine  if  it 
provided  a  logical  layout  for  the  technicians  to  reference  for  fault  isolation.  The 
contribution  of  the  automatic  fault  isolation  circuitry  to  the  troubleshooting  data  was 
examined.  The  following  troubleshooting  data  were  analyzed  for  completeness  and 
consistency  at  the  equipment,  subsystem,  and  system  levels:  (1)  troubleshooting  indices, 
(2)  mode  block  diagrams,  (3)  fault  isolation  and  detection  (FID)  block  diagrams,  (4)  fault- 
oriented  interconnector  diagrams  (FOIDs),  (5)  power  distribution  diagrams,  (6)  reference 
(schematic)  diagrams,  (7)  maintenance  control  ami  indicator  tables,  and  (8)  relay  and  lamp 
indices.  Problems  in  pa  'kaging  and  accessibility  of  the  data  were  noted  and  described. 


The  comprehensibility  of  the  OPs  was  examined  by  assessing  both  reading  difficulty 
and  text  layout  of  the  OP  material.  In  the  readability  analysis,  narrative  samples  from 
five  of  the  seven  major  OP  divisions  were  analyzed  using  the  "Writer's  Workbench"  text 
processing  package  developed  by  Bell  Telephone  Laboratories  (Macdonald,  Frase,  Ginsrich, 
&  Kennan,  1982).  Reading  grade  levels  (RGLs)  were  determined  according  to  the  method 
described  by  Kincaid,  Rodgers,  Fishburne,  and  Chissom  (1975). 

The  quasi-random  samples  of  materials  selected  from  the  various  NSSMS  manuals 
were  passages  of  continuous  text,  preferably  an  entire  paragraph,  approximately  100  to 
200  words  in  length,  without  tabular  material,  and  with  a  minimum  of  symbols  peculiar  to 
the  NSSMS  system.  The  number  of  samples  from  any  given  volume  was  approximately  in 
direct  proportion  to  the  amount  of  text  in  that  volume. 

Concurrently  with  the  human  factors  review,  fleet  personnel  were  sent  questionnaires 
and  interviewed  (see  appendix)  to  determine  if  general  usability  problems  exist  and  to 
identify  specific  problems  fleet  personnel  have  had  with  the  NSSMS  OPs.  Twenty 
technicians— 17  in  the  fire  control  technician  (surface  missile  fire  control)  (FTM)  rating 
and  3  in  the  gunner's  mate  (missiles)  (GMM)  rating- -responded  to  the  questionnaire.  Their 
experience  on  the  NSSMS  ranged  from  7  months  to  9-1/2  years.  Only  three  had  less  than  2 
years  experience,  while  nine  had  3  or  more  years  experience.  During  the  interview  phase, 
comments  were  obtained  from  nine  FTMs  and  two  GMMs.  All  of  these  had  at  least  2 
years  experience  on  the  NSSMS;  six  had  3  or  more  years. 

Because  of  the  problems  of  obtaining  field  data,  a  randomized  design  was  not 
possible.  Rather,  an  "as  available"  basis  was  used  to  obtain  both  questionnaire  and 
interview  data  from  both  Atlantic  and  Pacific  Fleet  personnel.  The  range  of  experience 
of  respondents,  however,  indicated  that  the  sample  is  representative  of  NSSMS 
technicians. 

Upon  completion  of  the  review,  a  plan  for  reformatting  the  NSSMS  technical 
documentation  was  defined.  This  plan  provides  for  OPs  that  satisfy  the  users'  needs  and 
are  consistent  with  the  user's  skills  and  knowledge,  while  making  maximum  use  of  current 
technical  content.  The  objectives  of  the  reformatted  OPs  are  to  provide  a  set  of 
documents  that  (1)  the  schools  can  use  readily  to  teach  troubleshooting  while  also 
providing  the  operational  information  needed  to  understand  how  the  NSSMS  functions,  (2) 
users  can  easily  access  for  troubleshooting  data,  and  (3)  the  NSSMS  Project  Office  can 
update  inexpensively  when  modifications  are  made  to  the  NSSMS. 


RESULTS  AND  DISCUSSION 


General  Problem  Areas 


Careful  examination  of  the  NSSMS  OPs  indicates  that  they  were  not  systematically 
developed.  Neither  the  separation  nor  the  relationship  of  the  operational  and 
troubleshooting  data  are  evident.  It  appears  that  an  original  core  of  technical  data  was 
developed  first  and  then  augmented  with  more  detailed  information.  What  emerges  is  a 
convoluted  mix  of  operational  and  troubleshooting  data  that  requires  making  a  choice 
between  teaching  the  system  or  teaching  how  to  troubleshoot.  A  more  integrated 
approach  would  use  the  operational  data  as  the  basis  for  imderstanding  how  the  system 
works,  while  at  the  same  time  using  the  troubleshooting  data  to  supplement  the 
operational  data  and  provide  the  user  with  the  necessary  information  to  operate  and 
maintain  the  system. 
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Confusion  between  understanding  how  the  system  operates  and  how  to  troubleshoot  it 
is  apparent  in  OP  4004,  the  technical  manual  that  should  provide  the  overall  structure  to 
the  technical  data.  Volume  1  presents  an  overall  operational  description,  while  Volume  2 
attempts  to  integrate  the  various  data  in  the  subsystem  OPs  to  provide  the  technician 
with  a  cohesive  structure  for  troubleshooting.  Unfortunately,  the  pattern  that  emerges  is 
not  what  technicians  use  to  fault  isolate  malfunctioning  components. 

Automatic  Fault  Isolation 


Built-in  test  (BIT)  and  FID  circuits,  which  are  an  integral  part  of  the  NSSMS 
circuitry,  are  designed  to  provide  the  technician  with  shortcuts  to  corrective  maintenance 
by  permitting  the  normal  troubleshooting  documentation  to  be  bypassed  for  some  faiiv 
Supposedly,  faults  not  isolated  by  the  BIT-FID  circuits  are  detected  during  the  NS'  S 
system  level  test  (i.e.,  the  daily  system  operability  test  (DSOT)).  Unfortunately,  * 

proportions  of  the  NSSMS  circuitry  covered  by  BIT-FID  circuits  and  DSOT  are  *i 
indicated.  To  compound  the  situation,  an  indication  of  failure  by  the  BIT-FID  circuit  i 
mean  that  either  the  operational  or  the  BIT-FID  circuitry  failed.  It  is  impossible  to  ) 
Therefore,  regardless  of  initial  fault  indication,  the  basic  troubleshooting  procedure  i:  ' 
same:  Start  with  the  indication  and  work  backwards  until  the  fault  is  discovered.  The 
technical  data  do  not  provide  the  user  with  the  information  necessary  to  go  directly  to  the 
suspected  fault. 

Trouble  Isolation 


In  OP  4004,  the  user  is  lead  to  believe  that  the  trouble  isolation  data  are 
systematically  organized  with  three  levels  or  sets  of  trouble  isolation  documentation: 
system,  subsystem,  and  equipment. 

At  the  system  level,  system  functional  diagrams  depict  development  and  transfer  of 
major  system  functions.  A  narrative,  detailed  functional  description  is  also  provided. 
Using  these  data  facilitates  checking  interface  signals  and  commands  between  the 
external  designation  source  (e.g.,  search  radar  and  NSSMS).  At  the  subsystem  level, 
design  of  the  trouble  isolation  documentation  is  supposed  to  provide  troubleshooting  down 
to  the  ship  replaceable  assembly  (SRA).  Thus,  at  this  level,  only  overall  fault  directories 
are  used  to  categorize  the  trouble  symptoms  and  reference  the  trouble  isolation  data  for 
specific  equipment. 

At  the  equipment  level,  fault  directories  from  either  the  system  or  subsystem  levels 
provide  additional  references  to  the  trouble  isolation  documentation  needed  to  isolate  the 
fault  to  a  defective  SRA  and  effect  necessary  corrective  action.  Descriptions  are 
provided  of  these  troubleshooting  data  (i.e.,  troubleshooting  indices,  mode  block  diagrams, 
FID  block  diagrams,  FOIDs,  power  distribution  diagrams,  reference  (schematic)  diagrams, 
maintenance  control  and  indicator  tables,  and  relay  and  lamp  indices)  but  not  of  their 
interrelationships. 

Figure  1  is  an  attempt  to  illustrate  the  various  relationships  among  these  data. 
Several  problems,  however,  must  be  noted.  The  usual  entry  points  for  troubleshooting  are 
either  DSOT  or  the  BIT  off-line  tests  and,  although  additional  information  sources  are 
available,  the  usual  troubleshooting  path  is  directly  from  the  indication  to  the  FOID.  For 
example,  consider  that  a  faulty  filter-rectifier  assembly  (15/16A4)  lights  the  AMPLIFIER 
ARC  Indicator  on  the  control  monitor.  Technicians  using  the  trace- bade  method  would 
begin  with  FOID  Figure  5-231,  sheet  3  of  3  and  trace  back  through  sheet  3  to  sheet  2  and 
eventually  to  sheet  1.  From  there,  another  book,  the  power  distribution  diagrams,  has  to 
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Figure  1.  Overview  of  the  NSSMS  troubleshooting  data  categories. 


be  consulted  to  find  Figure  5-411,  sheet  6.  Tracing  back  from  sheet  6  to  sheet  5  leads  back 
to  the  first  book  to  FOID  Figure  5-239,  sheet  7  of  14.  Tracing  back  from  sheet  7  to  sheet 
6  and  finally  to  sheet  5,  the  filter-rectifier  assembly  can  be  isolated  as  the  source.  For 
additional  technical  information  on  that  assembly,  however,  the  technician  must  consult  a 
third  book,  the  reference  diagrams,  to  find  the  schematic,  Figure  6-110.  Thus,  technicians 
had  to  refer  to  nine  pages  of  data  not  in  a  direct  sequence  and  in  three  books.  Since  only 
figure  numbers  are  referenced,  technicians  must  know  which  figure  numbers  are  in  which 
book  to  avoid  having  to  search  through  several  books  to  locate  a  specific  figure.  With  a 
more  complicated  problem,  the  laborious  process  of  signal  tracing  through  many  pages  can 
easily  become  confusing  and  it  is  easy  for  the  technicians  to  lose  their  place  going  from 
one  sheet  to  another.  With  many  sheet  and  figure  references  involved,  a  typographical 
error  can  easily  lead  the  technicians  astray. 

In  addition  to  FOIDs,  mode  and  FID  data,  both  in  narrative  and  diagrams,  are 
available  in  the  OPs  to  facilitate  the  troubleshooting  process.  The  mode  and  FID 
diagrams  are  presented,  at  both  the  general  and  detailed  levels,  as  separate  types  of  data 
but  integrated  with  each  other.  Both  are  important  when  troubleshooting  beyond  the  BIT- 
FID  circuitry  and  are  usually  referenced  whenever  a  FOID  is  referenced.  However,  the 
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mode  and  FID  data  are  in  the  operational  section  of  the  technical  manuals  and  are 
essentially  redundant  with  the  FOIDs.  Their  main  differences  are  that  (1)  the  FOIDs 
present  all  the  relevant  data  together,  based  on  indicator  entries  instead  of  first 
presenting  the  circuitry  associated  with  a  particular  mode  or  FID  circuit  and  (2)  they 
identify  test  points,  which  the  mode  and  FID  diagrams  do  not.  The  end  result  is  15,000  or 
20,000  pages  of  data,  of  which  technicians  only  use  about  5,000.  While  the  mode  diagrams 
may  be  useful  in  a  school  environment,  time  in  that  environment  is  limited.  Since  the 
school  is  designed  to  teach  troubleshooting,  use  of  the  FOIDs  is  emphasized  over  the  mode 
and  FID  data,  even  though  narrative  explanations  are  available  for  the  mode  and  FID 
diagrams  and  not  for  the  FOIDs. 

Although  the  maintenance  philosophy  is  to  troubleshoot  to  the  ship  replaceable 
assembly  (SRA),  inconsistencies  i».  the  data  do  not  always  permit  easy  identification  of 
the  SRA.  For  example,  the  master  oscillator  power  supply  (MOPS)  filament  power  supply 
(1 1/12A1A1A4)  has  been  identified  as  an  SRA  in  some,  but  not  all,  of  the  documentation. 
The  FOID  (Figure  5-233),  power  distribution  diagrams  (Figure  5-411),  and  the  additional 
information  on  the  schematic  (Figure  6-384)  do  not  identify  the  11/12AIA1A4  as  an  SRA 
that  can  be  replaced.  The  mode  and  FID -diagrams  (Figures  2-47  and  2-127)  illustrate  the 
master  oscillator  power  supply  (11/1 2 A 1 A 1 )  as  an  identifiable  unit  and  provide  no  further 
information. 

Organization  and  Packaging 


One  of  the  more  basic  problems  with  the  NSSMS  OPs  is  that  they  are  not  designed  lor 
the  technicians'  use.  One  might  assume  that  a  logical  breakdown  of  the  NSSMS  would 
include  a  system-level  document,  a  document  for  each  of  the  subsystems,  and,  within  each 
of  those  documents,  operational  and  troubleshooting  data.  In  fact,  there  are  a  system- 
level  document  and  subsystem  documents.  Unfortunately,  there  are  also  equipment 
documents  organized  at  the  subsystem  level.  For  example,  the  computer  complex  and 
signal  data  converter,  parts  of  the  fire  control  subsystem,  are  presented  in  separate 
documents. 

Each  document  consists  of  one,  two,  three,  or  five  volumes  with  each  volume 
consisting  of  one  to  ten  parts.  Why  such  a  confusing  organization  was  selected  is  not 
obvious  or  explained.  To  add  further  obfuscation,  some  of  the  classified  parts  contain 
large  sections  of  unclassified  material.  Apparently,  continuity  was  more  important  than 
was  convenient  user  access. 

Sizing  is  also  a  problem.  While  some  material  was  easily  packaged  into  an  8-1/2-  by 
11-inch  book,  the  numerous  oversized  drawings  necessitated  the  use  of  many  11-  by  16- 
inch  books.  Unfortunately,  these  books  have  no  identifying  marks  on  their  spine  and  the 
only  way  to  store  them  is  one  on  top  of  the  other. 

One  might  have  assumed  that,  with  so  many  books  (48)  and  the  problems  arising  from 
different  security  classifications  and  sizes,  the  designer  would  have  provided  extensive 
indexing  to  locate  the  numerous  types  of  technical  data.  Unfortunately,  this  was  not 
done.  In  fact,  only  a  single  table  of  contents  is  provided  in  any  given  volume.  Thus,  to 
locate  a  drawing  in  Part  7,  the  table  of  contents  in  Part  1  has  to  be  consulted  to  obtain  the 
page  number.  This  is  particularly  troublesome  for  OP  4005,  Volume  1,  because  Part  1  is 
classified  and  four  of  the  remaining  parts  are  not. 
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Readability  Analysis 


Analyses  to  determine  reading  grade  levels  (RGLs)  do  not  take  into  account  the 
reader's  overall  life  experiences  that  have  brought  the  reader  into  contact  with  words  and 
ideas  that  have  no  true  correlation  with  RGL.  In  addition,  the  vocabulary  words  peculiar 
to  an  individual's  area  of  interest  or  vocation  are  learned  independently  of  RGL  and, 
therefore,  do  not  contribute  to  the  difficulty  of  written  material  regardless  of  length. 
Such  words  as  superheterodyne,  oscilloscope,  or  ionospheric  inflate  the  RGL  because  they 
consist  of  so  many  syllables  but  are  not  difficult  to  read  for  those  in  the  field  of 
electronics.  Therefore,  the  concept  of  RGL  should  be  interpreted  cautiously  and  not 
literally.  To  say  that  NSSMS  technicians  with  an  RGL  of  12  cannot  read  technical 
material  with  an  RGL  of  14.5  is  to  interpret  the  facts  much  too  literally. 

Table  1  provides  the  number  of  words  and  sentences,  the  average  sentence  and  word 
length,  the  longest  sentence,  and  the  RGL  for  the  passages  with  the  lowest  and  the 
highest  RGL  from  each  manual  and  the  overall  RGL  for  all  passages  from  that  manual. 
As  can  be  seen  from  Table  1,  passages  analyzed  in  the  total  NSSMS  documentation  ranged 
from  an  RGL  of  6.9  (OP  4006,  paragraph  2-453)  to  21.3  (OP  4007,  paragraph  2-180).  For 
comparison,  these  two  passages  are  reproduced  in  Figure  2.  The  nature  of  the  words  used 
in  the  two  passages  does  not  vary  extensively.  However,  the  second  sentence  in  Figure  2b 
places  a  very  great  burden  on  the  reader  in  maintaining  the  train  of  thought  because  it  is 
67  words  in  length.  That  is,  the  critical  characteristic  of  these  passages  for  readability 
seems  to  be  sentence  length  and  not  word  length  and  its  associated  issue  of  the  use  of 
technical  terminology. 

Sometimes,  however,  word  length  is  a  problem  that  is  not  tied  to  the  technical 
language  of  the  system.  Take,  for  example,  paragraph  1.2  of  OP  4005,  which  reads: 

1.2.  Guided  Missile  Fire  Control  System  MK  91  Mods  0  and  1, 
hereinafter  referred  to  as  the  GMFCS,  together  with  Guided  Missile 
Launching  System  MK  29  Mod  0,  hereinafter  referred  to  as  the 
GMLS,  and  eight  Guided  Missiles  RIM-7H,  hereinafter  referred  to  as 
the  missile,  comprise  the  NATO  5EASPARROW  Surface  Missile 
System  MK  57  Mods  0  and  I,  hereinafter  referred  to  as  the  NSSMS. 

The  NSSMS  and  GMFCS  Mod  0  is  equipped  with  a  single  radar;  the 
Mod  1  is  equipped  with  dual  radars. 

This  passage  hao  an  RGL  of  20.8,  an  average  sentence  length  of  41.5  words,  and  an 
average  word  length  of  4.57  characters.  The  information  in  this  paragraph  is  relatively 
straightforward  and  simple,  but  its  "legalistic"  language  inflates  the  RGL  considerably  and 
probably  confuses  some  of  the  personnel  who  must  read  and  understand  the  paragraph. 
Such  terminology  has  no  place  in  Navy  technical  manuals. 

The  solution  to  the  problem  of  such  writing  may  be  relatively  simple.  In  cases  where 
the  RGL  is  too  high,  an  experienced  technical  writer/editor  should  be  able  to  lower  the 
RGL  as  needed. 


It  should  be  noted  that  there  is  no  evidence  to  show  that  the  usability  or  compehensi- 
bility  of  technical  writing  has  any  one-to-one  correspondence  with  its  measured  reada¬ 
bility  or  RGL.  However,  warning  flags  should  fly  when  the  computed  RGL  reaches  the 


Table  1 


Readability  Data  for  Selected  NSSMS  Passages 


Sentences 

Words 

Sample 

RGLa 

No. 

Avg.  Length 
( words) 

Longest 

(words) 

No. 

Avg.  Length 
(characters) 

OP  4004 

Lowest  RGL 

10.0 

6 

20.2 

22 

121 

4.68 

Highest  RGL 

19.4 

4 

31.5 

56 

126 

5.32 

Overall 

13.4 

62 

22.2 

56 

1377 

5.04 

OP  4005 

Lowest  RGL 

9.5 

5 

15.6 

26 

78 

4.54 

Highest  RGL 

20.8 

2 

41.5 

63 

83 

4.57 

Overall 

14.3 

51 

23.5 

63 

1200 

5.11 

OP  4006 

Lowest  RGL“ 

6.9 

8 

21.3 

44 

170 

4.09 

Highest  RGL 

18.9 

4 

32.0 

36 

128 

5.30 

Overall 

12.8 

378 

24.0 

49 

3785 

4.81 

OP  4007 

Lowest  RGL 

7.4 

7 

18.0 

36 

126 

4.35 

Highest  RGLC 

21.3 

2 

48.0 

67 

96 

4.70 

Overall 

12.7 

55 

25.2 

67 

1388 

4.83 

OP  4036 

Lowest  RGL 

9.9 

6 

20.3 

39 

122 

4.66 

Highest  RGL 

21.1 

2 

40.0 

42 

80 

5.47 

Overall 

12.6 

76 

23.1 

51 

1756 

4.97 

aRGL= reading  grade  level. 

bSee  Figure  2a  for  paragraph  2-453. 


cSee  Figure  2b  for  paragraph  2-180. 
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2-453.  Controlling  the  RAM  during 
STAA.  In  the  DECODER  (B3)  the 
RAM  write  enable  is  generated  by  an 
AND  gate  on  receipt  of  CROM  BIT 
10,  CROM  BIT  11,  CLOCK  3,  and  4 
MHZ  CLOCK.  THis  input  combina¬ 
tion  produces  a  125-nanosecond  pulse 
in  the  second  half  of  the  third  4- 
MHZ  CLOCK  period.  In  the  RAM 
(A7)  and  DATA  RAM  logic,  the 
LOAD  RAM  ADDRESS  COUNTER 
pulse  is  generated  by  a  NAND  gate 
on  receipt  of  CROM  BIT  11,  Clock  2, 
and  4  MHZ  CLOCK.  This  input 
combination  produces  a  125-nano¬ 
second  pulse  in  the  second  half  of 
the  second  4-MHZ  clock  period.  The 
pulse  sets  the  address  counter  to  the 
value  of  CROM  BIT  0-7.  The  RAM 
ADDRESS  COUNTER  output  is  the 
ADDRESS  input  to  the  RAM.  There¬ 
fore,  it  can  be  seen  that  MUX  OUT¬ 
PUT  BITS  0-7  are  received  by  the 
RAM  at  the  start  of  the  instruction, 
the  RAM  is  addressed  in  the  second 
clock  period  and  the  word  is  written 
into  the  RAM  in  the  third  clock 

,  period. 

!  a.  OP  4006,  paragraph  2-453: 

RGL  =  6.9. 


2-180.  Although  for  each  channel  there 
are  only  two  address  registers,  one 
associated  with  inputs  and  one  with 
outputs,  the  output  data  may  be  normal 
data  or  external  function  code  words. 
Therefore,  for  each  channel,  three 
separate  address  blocks  must  be  used, 
each  according  to  the  type  of  data  to  be 
transmitted  (i.e.,  input  data,  normal 
output  data,  or  output  of  external 
function  code  words)  and  a  command 
must  be  accompanied  by  one  of  nine 
output  device  addresses  (three  for  each 
channel)  to  identify  both  the  channel  and 
type  of  data  to  which  the  new  address 
block  refers. 


b.  OP  4007,  paragraph  2-180: 
RGL  =  21.3. 


Figure  2.  Selected  NSSMS  passages  with  the  lowest  and  highest  RGL. 


proportions  found  in  these  manuals.  Assume  for  a  moment  that  technicians  can  read  any 
material  written  at  the  high  school  graduate  level  (i.e.,  RGL=12),  which  may  or  may  not 
be  a  safe  assumption.  Table  2  illustrates  what  this  cutoff  grade  indicates  for  the  samples 
from  the  NSSMS  manuals.  If  this  assumption  is  correct  (and  the  median  RGL  of  Navy 
personnel  in  general  is  closer  to  10  than  12),  technicians  have  some  problem  in  compre¬ 
hending  60  percent  or  more  of  the  NSSMS  technical  data. 

The  major  issue  in  this  effort  is  the  improvement  of  comprehensibility  and  not  just 
the  measurement  of  readability.  Because  of  the  issues  involved  in  the  learning  of  the 
language  of  one's  work  and  social  milieu,  RGL  probably  does  not  adequately  reflect  the 
comprehensibility  of  material.  Rather,  it  is  only  am  indication  as  to  whether  or  not  the 
material  is  comprehensible.  It  is  safe  to  assume  that  RGL  underestimates  the  reading 
potential  of  the  incumbent  technician.  Unfortunately,  there  is  no  convenient  method  of 
assessing  the  comprehensibility  of  material  as  the  readability  formula  assesses  read¬ 
ability.  The  only  true  measure  of  whether  or  not  material  is  comprehensible  is  to  have 
the  technician  read  the  material  and  perform  the  task  it  prescribes.  If  the  task  is 
performed  in  an  efficient  and  effective  manner,  it  is  safe  to  say  that  the  technician 
comprehends  the  material. 


Table  2 


Percentage  of  Samples  From  NSSMS  Technical  Manuals 
with  RGLs  Below  and  Above  12 


Manual 

Percentage  of  Samples 

RGL  =  Below  12 

RGL  =  Above  12 

OP  4004 

38 

62 

OP  4005 

09 

91 

OP  4006 

28 

72 

OP  4007 

31 

69 

OP  4036 

29 

71 

Organization  is  another  characteristic  of  written  material  that  is  directly  related  to 
comprehensibility.  There  is  no  objective  index,  however,  indicating  how  well  the  material 
is  organized.  Some  aspects  of  organization  are  (1)  indexing  so  that  the  location  of  all 
major  topics  can  be  readily  located  in  the  text,  (2)  logical  flow  of  paragraphs  within 
sections  and  of  ideas  within  paragraphs,  and  (3)  "formatting"  so  that  single  units  (e.g., 
paragraphs)  contain  single  major  ideas. 

In  general,  the  NSSMS  documentation  suffers  from  a  lack  of  sufficient  breakdown  of 
textual  information.  That  is,  many  paragraphs  attempt  to  cover  an  entire  sequence  of 
ideas  that,  with  some  forethought  and  a  more  suitable  format,  could  be  offered  in  a  more 
organized  manner.  Take,  for  example,  paragraph  2-184  from  OP  4005,  presented  in  Figure 
3a,  which  was  picked  basically  at  random.  This  paragraph  conveys  a  great  deal  of 
information  without  benefit  of  the  pauses  provided  by  a  new  paragraph.  (This  is  not  an 
isolated  instance  but  is  the  rule  rather  than  the  exception.) 

Note  the  difference  in  the  ease  with  which  the  information  can  be  accessed  if 
suitable  breaks  are  provided  as  shown  in  Figure  3b.  No  effort  was  made  to  perform  other 
changes  to  make  this  material  more  readable.  Further,  there  is  no  evidence  that  the 
revised  version  is  any  more  comprehensible  than  the  original.  But  intuitively,  the  revision 
appears  to  be  more  readily  understandable  than  the  original  version. 

Specific  Problem  Areas 

OP  4004 


1.  Volume  1.  Volume  1  does  not  refer  the  reader  to  the  more  detailed  descriptions 
in  the  subsystem  documents.  Part  1  is  classified  but  contains  unclassified  reference 
information  (e.g.,  Tables  1-1  to  1-4)  that  may  be  more  useful  as  an  unclassified  appendix. 
Part  2  is  an  oversized  volume  with  excessive  white  space:  Two- thirds  of  the  pages 
contain  text  and  tables  on  only  one-half  of  the  page.  It  contains  oversized  unclassified 
drawings  that  could  be  reduced  and  turned  to  fit  on  an  8-1/2-  by  11-inch  page.  Volume  1 
does  not  provide  adequate  information  on  how  the  entire  OP  suite  is  organized  or  how  to 
use  its  components,  although  it  is  a  general  description  document. 

2.  Volume  2.  Volume  2  presents  a  confusing  array  of  information  on  the 
organization  of  the  NSSMS  fault  isolation  data.  System  functional  diagrams  are 
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presented,  but  the  narrative  explaining  the  diagrams  is  in  a  separate  book,  Part  1  of 
Volume  1.  In  the  sample  problems  described  in  Volume  2,  the  system  performance-codes 
fault  directory  has  much  of  the  same  information  as  the  performance  fault  trouble¬ 
shooting  indices  of  the  subsystem  OPs.  The  sample  problem  implies  that  the  trouble¬ 
shooting  procedure  will  ensure  the  fault  is  isolated,  but  that  the  FOIDs  have  to  be 
consulted  to  determine  if  a  maintenance  requirement  card  (MRC)  is  available  to  describe 
the  corrective  maintenance  procedures.  A  more  concise  approach  would  have  deleted  the 
redundant  information  and  reduced  the  amount  of  page  turning. 

OP  4005 

1.  Volume  1.  Part  1  repeats  the  introductory  material  of  OP  4004.  Although  Part  1 
is  supposed  to  be  the  introductory  document  for  the  NSSMS  fire  control  system,  it 
contains  no  information  on  the  signal  data  converter  or  the  low-light  level  television. 
Part  1  is  a  classified  document  that  contains  the  mostly  unclassified,  general  functional 
narrative  description  that  supports  the  function  description  diagrams  presented  in  Part  2. 

Part  3  is  also  classified  and  contains  the  detailed  functional  description. 
However,  the  mode  diagrams  that  this  descriptive  narrative  supports  are  in  Part  4. 

Part  5  is  the  narrative  that  supports  the  general  block  and  detailed  FID  diagrams 
in  Part  6.  While  it  may  be  important  to  know  that  there  are  FID  circuits  and  operational 
circuits,  it  is  not  readily  apparent  why  the  FID  is  presented  separately  from  the  mode 
circuitry.  To  know  how  both  the  FID  and  mode  circuits  are  integrated  in  the  NSSMS,  the 
reader  has  to  refer  back  and  forth  between  six  different  books. 

2.  Volume  2.  Volume  2  has  ten  parts  and  is  the  primary  source  for  troubleshooting 
data  for  the  fire  control  subsystem.  It  contains  at  least  ten  categories  of  useful 
troubleshooting  data,  which  are  very  difficult  to  use  because  of  inconsistent  presentation 
methods  and  poor  organization  schemes. 

Troubleshooting  flowcharts  are  described  in  Part  1  but  without  location  informa¬ 
tion.  The  format  of  fault  directories  for  each  equipment  varies  from  two  to  three 
columns  while  that  of  troubleshooting  indices,  which  the  fault  directories  reference, 
varies  from  three  to  four  columns.  Column  headings  differ  from  equipment  to  equipment 
(see  Figure  4).  Those  on  the  BITs  differ  from  those  on  the  MRCs.  Maintenance  turn-on 
procedures  are  included  only  for  selected  equipment.  The  format  of  the  relay  and  lamp 
indices  varies  from  four  to  six  columns  (see  Figure  4).  The  maintenance  controls  and 
indicators  are  described  as  reference  indices  and  illustrated  in  another  book.  References 
to  these  illustrations  are  sometimes  omitted. 

The  FOIDs  are  the  basic  data  used  to  troubleshoot.  Unlike  the  other  trouble¬ 
shooting  data,  the  FOIDs  are  organized  separately  by  cabinet  and  placed  into  individual 
books.  The  ship  replaceable  items  index  in  each  FOID  book  lists  much  of  the  same 
information  listed  in  the  parts  breakdown  in  Volume  3.  The  power  distribution  diagrams, 
on  the  other  hand,  are  separate  from  both  the  FOIDs  and  the  other  troubleshooting  data 
and  are  organized  into  a  single  volume.  The  reference  diagrams  (mostly  schematics)  are 
organized  by  drawing  number  and  not  by  cabinet.  As  the  schematics  are  assigned  new 
drawing  numbers,  a  blank  page  with  a  reference  to  the  new  drawing  number  is  used  as  a 
marker  for  the  old  schematic,  thus  adding  time  and  confusion  to  searches  for  a  specific 
schematic. 


Figure  4.  Relay  and  lamp  indices:  Examples  of  inconsistent 
column  headings  and  entries. 


3.  Volume  3.  The  group  assembly  parts  list  is  poorly  organized  for  accessing 
information.  Since  it  is  ordered  by  ascending  part  number,  the  technician  has  to  move 
continually  from  one  spot  in  the  book  to  another  even  when  looking  for  parts  or  parts 
numbers  within  a  single  cabinet.  In  addition,  illustrations  identifying  components  are  not 
always  available.  Use  of  the  conventional  illustrated  parts  breakdown  would  make  it 
easier  for  the  technician  to  identify  both  the  part  and  part  number. 

OP  4006 

Volume  1  of  OP  4006  presents  some  of  the  same  problems  as  OP  4003.  Part  1 
contains  the  detailed  function  descriptions  that  support  the  mode  and  FID  diagrams  in 
Part  2. 

Since  OP  4005  covers  the  fire  control  subsystem  and  OP  4006  covers  the  launcher 
subsystem,  their  layout  and  organization  should  be  the  same.  Unfortunately,  they  are  not. 
For  example,  the  signal  is  more  difficult  to  trace  in  the  OP  4006  FOIDs  than  in  the  OP 
4005  FOIDs  since  the  test  points  are  not  as  easy  to  find.  As  a  result,  the  technician  hats  to 
use  the  reference  diagrams  (schematics)  in  conjunction  with  the  FOIDs  to  troubleshoot. 


/ 
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References  to  a  particular  schematic  includes  the  zone  coordinates, 
however,  have  no  zone  identification  marks. 

OP  4007 


The  dawings, 


The  foreword  to  OP  4007  indicates  its  unusual  layout  (see  Figure  5).  It  introduces 
five  volumes  to  OP  4007,  but  a  footnote  informs  the  reader  that  Volumes  2  and  4  have 
been  incorporated  into  Volumes  1  and  3,  respectively.  Therefore,  the  reader  is  confronted 
with  a  document  that  has  only  three  volumes,  which  are  numbered  Volumes  1,  3,  and  5. 


FOREWORD 


Ordnance  Publication  4007  (PMS/SMS)  explains  the  principles  of  operation  and  provides  Information 
necessary  for  operation  and  maintenance  of  Digital  Computer  MK  157  Mod  0. 


This  publication  constats  of  five  volumes: 

Volume  1  (PMS/SMS) 

Chapter  1 
Chapter  2 
Chapter  3 

*  Volume  2  (PMS/SMS) 

Chapter  4 

Volume  3  (PMS/SMS) 

Chapter  5 
Chapter  6 

*  Volume  4  (PMS/SMS) 

Chapter  7 
Chapter  8 

Volume  5  (PMS/SMS) 
Chapter  9 


•  Description  and  Operation 

-  Introduction 

-  Description 
'  Operation 

'  Scheduled  Maintenance 

-  Scheduled  Maintenance 

-  Trouble  Isolation  and  Reference  Diagrams 

-  Trouble  I  eolation 

-  Reference  Diagrams 

-  Corrective  Maintenance,  Yard  end  Tender  Repair 

-  Corrective  Maintenance 

-  Yard  and  Tender  Repair 

-  Replacement  Parts 

-  Replacement  Parts 


*  Chapter  4,  Volume  2  is  incorporated  into  VoLuaa  l. 
Chapters  7  and  8,  Volume  4  are  incorporated  into 
Volume  3  Part  4. 


Figure  5.  Foreword  for  OP  4007  describing  its  confusing  organization. 
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OP  4007  differs  greatly  from  OP  4005  and  OP  4006  in  format  and  level  of  detail 
presented.  The  text  of  OP  4007  is  presented  in  a  double  column  instead  of  in  a  single 
column  and  is  in  smaller  type.  Rather  than  having  the  illustrations  in  separate  books,  the 
illustrations  for  OP  4007  are  integrated  with  the  narrative.  Instead  of  having  general 
block,  detailed  block,  and  FID  diagrams  in  separate  books,  the  block-text  diagrams  are 
integrated  with  the  narrative  to  provide  the  reader  with  an  operational  understanding  of 
the  computer. 

Fault  isolation  data  are  limited  to  fault  response  tables.  In  other  words,  the  fault 
indices,  troubeshooting  indices,  and  FOIDs  that  facilitate  troubleshooting  on  the  other 
parts  of  the  fire  control  subsystem  are  not  used  for  the  computer.  The  technician  has  to 
use  a  different  troubleshooting  strategy  that  requires  starting  at  the  beginning  of  the 
fault  response  table  and  proceeding  until  a  condition  fails  and  the  faulty  component  is 
identified. 

OP  4036 

Like  OP  4007,  OP  4036  is  a  complete  departure  from  the  formats  used  in  both  the 
fire  control  and  launcher  subsystem  volumes  (OP  4005  and  OP  4006).  Test  procedures  are 
embedded  in  the  narrative.  The  coordinate  system  used  to  locate  areas  on  the  drawings  is 
completely  different  from  that  used  on  other  fire  control  subsystem  drawings  (see  Figure 
6 );  neither  system  complies  with  the  specification.  Function  diagrams  are  used  instead  of 
detailed  description  diagrams.  FID  diagrams  are  not  used.  Connection  diagrams  are  used 
instead  of  FOIDs.  Only  partial  reference  designations  are  used.  Schematics  are  also  in  a 
different  format. 

OP  4053 

OP  4053  is  similar  in  format  to  OP  4007  and  in  organization  to  OP  4036.  It  does  not 
resemble  OP  4005  or  OP  4006.  The  illustrations  are  integrated  with  the  text  with  detailed 
block  diagrams  supplemented  with  schematics.  Its  organization  and  integration  of 
technical  material  are  the  best  of  the  NSSMS  documents. 

System  Evaluation  and  Test  Manuals 

The  system  evaluation  and  test  (SEAT)  manuals  consist  of  an  assortment  of  manuals 
whose  format  differs  from  manual  to  manual  and  even  from  chapter  to  chapter. 
Incorporating  off-the-shelf  manuals  as  NSSMS  documents  is  economical  for  the  acquisition 
cycle  costs  but  may  affect  the  life-cycle  cost  of  the  NSSMS  greatly.  The  organization  of 
the  appendices  make  it  extremely  difficult  to  access  information  (e.g.,  there  are  three 
Appendix  3s,  each  with  its  own  table  of  contents).  Because  these  are  vendor  manuals, 
much  of  the  information  has  no  relationship  to  SEAT  operation  and  repair  (e.g.,  the  power 
supply  information  includes  mounting  instructions,  which  are  unnecessary  since  the  power 
supplies  have  already  been  mounted  into  the  SEAT  cabinet). 

Illustrations 


Overall,  one  of  the  more  inconsistent  and  confusing  areas  for  accessing  troubleshoot¬ 
ing  information  is  the  referencing  system  to  go  from  one  drawing  to  another.  Figure  7 
presents  examples  of  some  of  the  drawing  reference  schemes  used  in  the  OPs. 


NAVSEA  OP  4005  (PMS/SMS)  VOLUME  2 


NAV.SE A  OP  400S  (PMb/SMS)  VOLUME  l 


NAVSEA  OP  4036  <  PMS/SMS)  VOLUME  2 


NAVSEA  OP  4006  (PMS/SMS)  VOLUME  2 


u  *t 


NAVSEA  OP  40M  (PMS/tMt) 


Figure  7.  Examples  of  different  referencing  system  used  to  trace  a  signal  from  one  C° 
to  another. 
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In  OP  4005,  reference  to  the  same  FOID  is  made  by  grid  coordinates  only.  To  refer 
to  another  fire  control  drawing,  both  the  figure  number  and  grid  coordinates  are  given. 
References  to  OP  4036  include  OP  number,  full  figure  title  and  drawing  number,  and  grid 
coordinates,  but  no  figure  or  page  numbers.  References  to  OP  4006  include  only  the  OP 
number  and  the  full  figure  title. 

In  OP  4006,  the  only  difference  from  the  reference  system  used  in  OP  4005  is  the 
reference  to  OP  4005.  In  OP  4006,  the  drawings  are  identified  as  FOIDs. 

Being  a  different  design  from  either  OP  4005  or  OP  4006,  OP  4036  has  a  different 
reference  system.  References  to  either  the  same  drawing  or  other  drawings  within  OP 
4036  include  the  use  of  a  "D"  number  to  identify  the  figure,  the  sheet  number  of  the 
figure,  and  the  grid  coordinates.  References  to  drawings  in  other  manuals  use  only  the 
phrase  "to/from  peripheral  equipment."  The  technicians  have  to  find  the  signal  path.  In 
addition,  drawings  of  similar  cards  are  not  repeated.  Users  have  to  identify  the  card  they 
are  troubleshooting  and  than  refer  to  a  table  of  values  to  determine  what  values  apply  in  a 
given  situation. 

In  OP  4053,  the  schematics  provide  no  references  between  drawings.  The  technician 
is  forced  to  follow  the  signal  by  name. 

Fleet  Input 

Fleet  Questionnaires 

Packaging  and  Organization.  The  technicians  did  not  like  the  intermixing  of  the  sizes 
of  the  volume  (8- 1/2"  x  11"  vs.  11"  x  16").  Many  responded  that  experience  was  needed  to 
understand  the  organization  of  data  within  the  OPs.  They  wanted  the  troubleshooting 
information  to  be  organized  by  cabinet,  instead  of  providing  all  the  fault  indices,  etc.  in  a 
single  book,  while  the  FOIDs  are  organized  by  cabinet  and  placed  into  separate  books  with 
one  for  each  cabinet  or  cabinet  group.  A  systems  OP  could  illustrate  the  interconnections 
among  the  cabinets.  With  the  schematics  and  parts  information  for  all  the  cabinets  also 
packaged  together,  the  troubleshooting  data  are  too  spread  out  for  the  technicians  to  use 
efficiently. 

Technicians  expressed  frustration  at  the  inconsistent  level  of  detail  of  the  trouble¬ 
shooting  data.  If  more  detailed  schematics  for  specific  ship  replaceable  assemblies  (e.g., 
power  supplies)  were  available,  the  technicians  could  identify  and  replace  some  of  the 
failed  components  when  SRAs  are  unavailable. 

Using  the  OPs  for  Troubleshooting.  Technicians  stated  that  it  was  not  always  clear 
where  the  troubleshooting  process  should  begin  and  sometimes  the  information  flow  within 
the  OPs  would  mislead  them.  Necessary  branching  information  was  absent  when  one  path 
failed  to  isolate  the  fault.  All  the  technicians  stated  that  good  information  was  not 
available  for  troubleshooting  multiple  faults.  OP  4036  was  cited  as  an  example  for  being 
very  hard  to  find  a  starting  place  for  troubleshooting  the  signal  data  converter.  The 
technicians  troubleshooting  the  computer  disliked  the  way  in  which  the  troubleshooting 
data  were  organized  in  OP  4007. 

Amount  of  Data.  In  response  to  questions  concerning  the  amount  of  data  in  the  OPs, 
the  technicians  wanted  a  more  detailed  breakdown  to  be  able  to  identify  the  failed 
components.  All  agreed  that  insufficient  guidance  was  given  for  evaluating  specific 
waveform  patterns.  The  description  of  what  the  waveform  should  look  like  is  incomplete, 
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incorrect  waveforms  are  presented,  or  the  waveforms  are  inadequately  explained.  All 
would  like  a  narrative  to  explain  the  FOIDs. 


Specific  Problems.  The  specific  problems  cited  by  the  20  technicians  who  responded 
to  the  questionnaire  are  listed  below: 

1.  In  OP  4006,  Volume  1,  detailed  functional  descriptions  are  often  difficult  to 
follow  due  to  the  necessity  to  refer  back  and  forth  through  the  diagrams  in  Part  2  while 
trying  to  follow  the  functional  descriptions  in  Part  1  (e.g.,  the  initialization  circuits, 
Figure  2-21).  Tracing  the  ship's  power  requires  seven  sheet  changes--from  1-6-5-1-5-1-6- 
1;  the  system  control  power,  four  sheet  changes — from  1-8-13-1;  and  the  stow  lock 
operation,  seven  sheet  changes--from  1-9-7-11-1-12-1. 

2.  In  OP  4006,  Volume  2,  Part  2,  Figure  5-123,  voltages  must  be  read  at  various  test 
points  when  troubleshooting.  However,  actual  voltages  rarely  equal  the  assigned  BITE 
test  mode  value  listed  on  the  FOID  and  no  tolerances  are  given  to  indicate  an  acceptable 
voltage  range.  Thus,  technicians  must  guess  if  a  circuit  is  functioning  properly  when,  in 
fact,  a  lower  or  higher  voltage  might  be  an  indication  of  other  circuit  problems.  Due  to 
the  lack  of  replacement  units  for  the  servo  electronics,  troubleshooting  is  very  limited 
and  diagnosis  cannot  be  verified  by  replacing  suspected  bad  cards.  Therefore,  more  time 
must  be  spent  comparing  input  and  output  voltages  of  circuit  components  to  determine 
the  exact  cause  of  system  failures.  Listing  voltage  tolerances  under  operational 
conditions  at  the  test  points  on  the  FOID  would  facilitate  trouble  isolation  and  eliminate 
guesswork.  Tolerances  are  needed  at  all  fault  test  points  in  FID  circuits  and  FOIDs. 

3.  If  the  waveforms  for  various  components  were  provided  on  the  FOIDs  instead  of 
on  supplemental  sheets,  troubleshooting  would  be  less  confusing. 

4.  Waveform  information  was  unclear  and  could  not  be  used  to  evaluate  the  "air 
signal"  in  the  radar  target  data  processor  (RTDP)  FOIDs. 

5.  To  troubleshoot  the  computer,  the  technician  has  to  start  at  the  beginning  of  the 
troubleshooting  table  and  continue  along  until  the  fault  is  found.  If  the  fault  is  not  found, 
the  technician  is  not  provided  with  amplifying  information  such  as  what  has  been  checked 
or  what  schematics  to  reference. 

6.  The  launcher  OP  (4006)  referenced  a  terminal  board  signal  that  was  not 
measurable  because  the  references  were  off  by  two  and  the  signal  that  was  referenced  did 
not  exist  on  the  referenced  terminal  board. 

7.  Reference  diagrams  do  not  contain  the  parts  number  information  needed  to 
identify  a  specific  component  (e.g.,  resistors  and  capacitors)  when  the  component  either  is 
missing  or  has  burned. 

8.  Following  a  signal  from  the  fire  control  system  OP  (4005)  to  the  launcher  OP 
(4006)  is  very  difficult  because  the  figure  number  for  the  continuation  of  the  signal  is  not 
provided. 


9.  The  RTDP  FOIDs  provide  insufficient  or  inaccurate  information  for  identifying 
specific  signals. 

10.  Not  enough  information  is  provided  at  specific  testpoints. 
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11.  Parts  breakdown  illustrations  do  not  identify  specific  components  that  have 
failed  (e.g.,  the  amplifier-oscillator  cooling  plate). 

12.  Not  enough  component  information  for  areas  of  the  transmitter  (e.g.,  repairing 
the  MOPS  rather  than  replacing  the  entire  assembly  for  a  minor  component). 

Fleet  Interviews 

Using  OPs  to  Troubleshoot  (Appendix,  Questions  1-7  and  11-14).  While  the  majority 
of  the  1 1  interviewees  stated  that,  after  initial  indication  of  a  fault,  they  run  a  BITE  test 
or  BIT  off-line  test  to  amplify  the  fault,  all  indicated  they  eventually  use  the  FOIDs  to 
troubleshoot.  Regardless  of  the  additional  troubleshooting  data  that  may  be  available  in 
the  mode,  FID,  or  schematic  diagrams,  the  basic  troubleshooting  procedure  was  to  start 
with  the  fault  light  and  trace  the  signal  backwards  from  that  light. 

Only  three  technicians  responded  that  they  use  the  fault  directories  to  help  locate  a 
FOID.  Their  responses  were  further  qualified,  however,  because  the  directories'  use  was 
limited  to  the  more  difficult  troubleshooting  problems  (e.g.,  the  continuously  monitored 
fault  codes  for  the  FOC  or  the  fault  light  codes  for  the  missile  control  unit). 

Nine  technicians  stated  that  troubleshooting  would  be  easier  if  a  narrative  was 
available  for  the  FOIDs.  Three  of  the  eight  technicians  who  had  not  used  the  mode 
diagrams  for  troubleshooting  agreed  that  the  mode  was  useful  for  teaching  the  system. 
Only  one  technician  stated  that  he  had  used  FID  diagrams  to  troubleshoot;  specifically,  to 
help  isolate  a  fault  in  the  transmitter. 

While  all  the  respondents  related  that  they  use  schematics,  only  four  use  the 
classified  OPs,  mostly  for  general  NSSMS  descriptive  information.  All  stated  that  they 
use  the  group  assembly  parts  list  to  identify  part  numbers.  Four  technicians,  however, 
stated  that  they  generally  use  the  ship  replaceable  items  indices  (in  the  FOID  books). 

Changing  the  OPs  (Appendix,  Questions  8-10).  All  the  technicians,  except  one,  stated 
that  a  better  indexing  scheme  would  make  the  OPs  easier  to  use.  Several  said  that,  if  the 
information  was  indexed  in  a  couple  of  different  ways,  a  starting  point  could  be  located 
faster. 

While  all  the  technicians  agreed  that  the  oversized  books  presented  a  storage 
problem,  only  two  expressed  an  interest  in  having  foldout  pages  in  place  of  oversized 
books. 

When  asked  about  specific  changes  to  the  OPs  (question  8),  the  following  responses 
were  given: 

1.  Add  narratives  to  the  FOIDs. 

2.  Reorganize  the  data;  there  are  too  many  different  types  of  data. 

3.  Reduce  the  bulk;  it  is  too  easy  to  get  lost  in  the  data  when  troubleshooting. 

4.  Periodically  review  the  OP  suite  aboard  each  ship  to  verify  that  the  latest 
changes  have  been  incorporated  correctly. 

5.  Provide  more  information  on  the  computer. 


6.  Redo  OP  4036  in  same  format  as  OP  4005  because  the  information  in  existing  OP 
4036  is  hard  to  follow. 

7.  Make  OPs  more  consistent;  the  launcher  OP  (4006)  is  not  as  detailed  as  the  fire 
control  system  OP  (4005)  and  requires  additional  referencing  to  the  schematics. 

8.  Both  OP  4007  and  OP  4036  need  to  be  improved  to  facilitate  fault  isolation. 


CONCLUSIONS 

1.  No  systematic  top-down  breakdown  approach  was  applied  to  the  organization  and 
development  of  the  NSSMS  technical  data  that  were  incorporated  into  the  OPs.  The 
evolution  of  the  OPs  produced  documents  that  present  functional  and  fault  isolation 
information  independently.  Thus,  the  technician  has  to  integrate  both  types  of 
information  to  build  a  complete  model  of  the  system. 

2.  Technicians  use,  at  most,  5,000  pages  of  the  15,000  to  20,000  pages  of  technical 
data  to  maintain  NSSMS.  Technical  information  relating  to  specific  equipment  is  spread 
out  within  the  OPs  in  a  manner  that  makes  it  extremely  hard  to  determine  if  all  available 
information  has  been  accessed.  Redundant  coverage  exists  in  some  areas,  while  gaps  exist 
in  others. 

3.  The  maintenance  philosophy  and  the  level  of  detail  of  the  technical  data  differ. 

4.  Trouble  isolation  almost  always  consists  of  tracing  back  from  a  fault  indicator 
through  many  pages  located  in  different  books. 

5.  No  narratives  exist  for  the  main  troubleshooting  data  source,  the  FOIDs. 

6.  Overall  packaging  and  organization  of  the  OP  documents  are  insensitive  to  the 
users'  access  requirements. 

7.  No  comprehensive  index  or  cross-indexing  of  the  technical  data  is  provided. 

8.  Only  15  of  the  48  plus  books  have  tables  of  contents. 

9.  The  overall  RGL  for  all  the  samples  analyzed  is  over  13.0  with  some  sections  as 
high  as  21.0.  Combining  the  high  RGL  with  the  poor  layout  and  organization  of  the  textual 
material  provides  a  set  of  documents  that  cannot  be  easily  accessed  for  specific 
information  topics. 

10.  The  overall  system  OP  (4004)  is  too  general  with  few  references  to  the 
subsystem  and  equipment  OPs. 

11.  To  understand  how  the  fire  control  subsystem  functions  in  different  modes,  the 
user  has  to  refer  back  and  forth  between  six  different  books. 

12.  Different  formats  are  used  to  present  similar  troubleshooting  data  (fault  tables, 
troubleshooting  indices,  etc.). 

13.  Technical  data  in  OP  4006  are  presented  at  a  different  level  of  detail  than  are 
data  in  OP  4005. 


14.  OP  4007  is  very  different  from  either  OP  4005  or  OP  4006. 

1 5.  OP  4036  is  also  very  different  from  either  OP  4005  or  OP  4006.  The  scheme  for 
illustration  references  differs  completely  from  that  used  in  the  rest  of  the  NSSMS  OPs. 

16.  The  format  and  organization  of  OP  4053  are  best;  as  a  result,  its  format  differs 
from  the  other  OPs. 

17.  SEAT  manuals  are  off-the-shelf  vendor  manuals  that  have  not  been  integrated 
with  the  other  NSSMS  OPs.  The  technical  data  in  the  SEAT  OPs  are  excessive,  far  more 
detailed  than  required  for  shipboard  maintenance,  and,  in  some  sections,  totally 
irrelevant. 

IS.  Grid  reference  schemes  for  illustrations  are  inconsistent  within  and  among  the 
NSSMS  OPs. 

19.  Fleet  personnel  have  problems  using  the  OPs.  They  prefer  to  have  a  more 
concise  and  usable  set  of  OP  documents.  For  some  areas,  more  detailed  information  is 
desired;  for  others,  less. 


RECOMMENDATIONS 

It  is  suggested  that  the  NSSMS  project  office  reformat  the  OPs  to  provide  the  fleet 
with  a  concise  set  of  systematically  designed  documents  that  (1)  can  be  used  by  both 
experienced  and  inexperienced  NSSMS  technicians,  (2)  fully  integrate  training 
requirements  and  the  PMS  procedures,  and  (3)  provide  fault  isolation  data  that  can  also  be 
used  in  the  school  to  provide  an  operational  understanding  of  the  system. 

To  accomplish  the  systematic  development  of  NSSMS  OPs,  the  original  task  analysis 
should  be  reexamined  to  ensure  that  the  function  analysis  has  been  completed  and  all 
personnel  requirements  have  been  identified. 

A  complete  functional  analysis  is  necessary  to  determine  if  the  required  source  data 
are  available  for  categorizing  equipment  types  and  complexity  and  for  matching  the  best 
technical  information  format  to  them.  From  the  personnel  requirements,  a  user 
description  can  be  developed  to  establish  the  level  of  detail  required  for  the  technical 
materials.  The  user  should  be  described  in  terms  of  job  relevant  skills,  knowledge, 
experience,  and  reading  ability.  The  user  description  provides  the  basis  for  the 
assumptions  concerning  the  technical  capabilities  of  the  NSSMS  technician  before  and 
after  training.  The  user  description  will  have  to  be  integrated  with  the  maintenance 
philosophy  to  ensure  that  an  NSSMS  technician  has  the  technical  data,  at  the  proper  level 
of  detail,  to  maintain  the  NSSMS. 

An  example  of  a  systematic  development  and  implementation  effort  for  revision  of 
the  NSSMS  technical  data  is  given  in  Figure  8.  To  ensure  the  timely  completion  of  this 
type  of  effort,  the  following  tasks  are  recommended. 
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Figure  8.  Example  of  development  process  for  revising  NSSMS  technical  data. 


1.  Organize  and  conduct  an  adhoc  committee  review  of  the  NSSMS  discrepancies. 
NAVPERSRANDCEN  will  organize  and  conduct  an  adhoc  meeting  with  PACFLT  and 
LANTFLT  personnel  to  discuss  the  discrepancies  with  the  existing  OPs.  Based  upon  the 
priorities  assigned  by  the  adhoc  committee,  NAVPERSRANDCEN  will  develop  a  specifica¬ 
tion. 


2.  Develop  specification  for  design  and  development  of  NSSMS  OPs.  NAVPERS¬ 
RANDCEN  will  combine  the  technical  input  from  the  Naval  Ship  Weapon  Systems 
Engineering  Station  (NSWSES)  with  tested  information  presentation  principles  to  design 
and  develop  a  specification  for  NSSMS  OPs.  The  specification  will  be  designed  to  provide 
a  technically  accurate  document  at  a  level  of  detail  that  is  usable  and  complete  for  a 
wide  range  of  users.  The  quality  assurance  section  will  include  the  guidance  needed  to 
ensure  that  the  revised  OPs  are  validated  and  verified  using  a  representative  user 
population.  NAVPERSRANDCEN  will  provide  the  necessary  support  to  the  NSSMS  project 
office  to  ensure  that  the  contractor  understands  all  parts  of  the  specification. 

3.  Attend  IPRs.  Upon  issuance  of  the  OP  development  contract,  NAVPERSRAND¬ 
CEN  will  provide  the  technical  support  required  at  in-process  reviews  (IPRs)  to  review  and 
evaluate  the  contractor’s  product  according  to  the  behaviorally-oriented  guidance  of  the 
specification.  In  coordination  with  NSWSES,  NAVPERSRANDCEN  will  determine  if  the 
contractor's  interim  products  are  leading  towards  a  document  that  is  technically  accurate, 
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complete,  and  usable  by  a  wide  range  of  users.  In  addition  to  the  analytical  review, 
preliminary  validations  will  be  conducted. 

4.  Preliminary  validations  and  problem  resolutions  using  fleet  personnel.  To  ensure 
that  the  end  document,  a  revised  set  of  N55M5  OPs,  is  technically  accurate,  complete, 
and  usable,  NAVPERSRANDCEN  will  conduct  preliminary  validations  using  the  contrac¬ 
tor's  interim  products.  Results  and  recommendations  for  changes  will  be  provided  to  the 
project  office  upon  completion  of  each  validation.  In  addition,  any  problems  that  arise 
during  the  development  cycle  regarding  the  usability  of  a  format  will  also  be  examined  in 
an  operational  environment. 
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FLEET  SURVEY  INSTRUMENTS 
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FLEET  SURVEY  INSTRUMENTS 

Questionnaire 

1.  Do  the  OPs  provide  you  with  enough  introductory  information  on  how  the  OPs 
are  organized? 

2.  Do  the  OPs  provide  you  with  enough  information  on  how  to  use  the  OPs  for 
troubleshooting? 

3.  Do  you  like  the  way  the  OPs  are  organized? 

Are  there  too  much  or  too  little  technical  data  in  the  OPs? 

3.  If  you  could  change  the  way  the  technical  data  are  presented  in  the  OPs,  how 
would  you  change  it? 

6.  Cite  a  specific  problem  you  have  had  with  the  OPs  (e.g.,  after  identifying  the 
initial  symptoms  for  the  malfunctioning  widget,  OP  400X  was  consulted,  which  led  to  a 
troubleshooting  table  that  provided  no  additional  data  on  the  problem;  i.e.,  it  was  a  dead 
end). 
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Structured  Interview  Questionnaire 


1.  How  do  you  use  the  OPs  to  troubleshoot? 

2.  Have  you  ever  used  the  fault  directories  or  troubleshooting  indices  in  4005  (4006) 
Volume  II,  Part  1  to  help  you  locate  a  specific  FOID? 

3.  If  FOIDs  had  a  narrative  description,  would  that  make  troubleshooting  any 
easier? 

4.  Have  you  ever  encountered  a  fault  where  you  used  the  mode  diagrams  to 
understand  what  was  supposed  to  be  happening  in  the  system? 

5.  Do  you  ever  use  the  FID  diagrams? 

6.  Do  you  ever  use  the  information  contained  in  the  CONFIDENTIAL  OPs? 

7.  Do  you  ever  use  the  schematics? 

8a.  Do  you  ever  use  the  Group  Assembly  Parts  List  (GAPL)  in  Volume  III? 

8b.  Would  an  IPB  be  easier  than  the  GAPL  to  use? 

9.  How  would  you  explain  to  another  technician  how  a  subsystem  works  (i.e.,  what 
parts  of  the  OPs  would  you  use,  if  any)? 

10.  Would  the  OPs  be  easier  to  use  if  each  part  had  a  table  of  contents  or  index? 

11.  Would  the  OPs  be  easier  to  use  if  the  oversized  books  were  composed  of  foldout 
pages? 

12.  What  would  you  like  to  see  added  to  or  deleted  from  the  OPs? 
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